Skip to content

Add MariaDB WASM support to the Playground CLI#3474

Draft
adamziel wants to merge 12 commits intotrunkfrom
adamziel/mariadb-wasm-cli
Draft

Add MariaDB WASM support to the Playground CLI#3474
adamziel wants to merge 12 commits intotrunkfrom
adamziel/mariadb-wasm-cli

Conversation

@adamziel
Copy link
Copy Markdown
Collaborator

@adamziel adamziel commented Apr 8, 2026

Summary

WordPress Playground has always used SQLite via a database integration plugin. This PR adds a second option: MariaDB 11.4 compiled to WebAssembly, running as an embedded server in the same Node.js process as PHP.

How it works

The architecture has three layers:

  1. MariaDB WASM binary — MariaDB 11.4's embedded server (libmysqld) compiled to WebAssembly via Emscripten. It exposes the MySQL C API (mysql_query, mysql_store_result, etc.) through cwrap(). The build lives in packages/php-wasm/mariadb-wasm-compile/ with a build.sh that clones MariaDB source on demand and applies patches for WASM: Aria runs in read-only mode (pagecache only, no threads), the timer thread is skipped, curses/GnuTLS/client tools are stripped, and servers_init respects --skip-grant-tables.

  2. @wp-playground/mariadb package — A JavaScript bridge (MariaDBBridge) wraps the C API with structured results, and a MySQL wire protocol server (startMySQLProtocolServer) implements enough of the MySQL client/server protocol over TCP for PHP's mysqli extension to connect. The protocol server handles handshake, COM_QUERY, COM_INIT_DB, COM_PING, COM_QUIT, COM_FIELD_LIST, and COM_SET_OPTION.

  3. CLI integration--database=mariadb flag (plus optional --mariadb-wasm-module=<path> override). When selected, the embedded MariaDB server starts before PHP worker threads spawn, the MySQL protocol server binds to a free TCP port, and WordPress gets DB_HOST=127.0.0.1:<port> in wp-config.php instead of the SQLite integration plugin.

Data storage

MariaDB's data files (MyISAM .MYD/.MYI/.frm) are stored on the host filesystem via Emscripten's NODEFS, in a mariadb-data/ subdirectory of the CLI's shared temp directory — the same mechanism used for /tmp, /wordpress, and /home.

WASM build patches

The MariaDB source needs several patches to compile and run in WASM:

  • Aria storage engine: Surgical init — maria_init() + pagecache allocation without control file, translog, recovery, or checkpoint threads. Aria still handles temp tables.
  • Timer thread: Skipped via #ifdef __EMSCRIPTEN__ — no setitimer/threading in WASM.
  • servers_init: Patched to respect --skip-grant-tables (was hardcoded to always load mysql.servers).
  • Build dependencies: Curses, GnuTLS, client tools stripped. PCRE2 gets the Emscripten toolchain. NODERAWFS enabled for build-time tools.
  • 64-bit integers: WASM_BIGINT=1 with explicit BigInt→Number conversion at the bridge boundary.

Running it

# Uses the in-repo pre-built module automatically
npx nx dev playground-cli -- server --database=mariadb

# Or with a custom module path
npx nx dev playground-cli -- server --database=mariadb \
  --mariadb-wasm-module=/path/to/mariadb.js

Open question: how to ship the mariadb.wasm binary

The pre-built mariadb.wasm is ~10MB. Currently it's committed to the repo in packages/php-wasm/mariadb-wasm-compile/dist/. That's not ideal for a few reasons — it bloats the git history and it would bloat npm install for everyone if it became a mandatory dependency.

Options to consider:

  1. Optional npm package — publish @php-wasm/mariadb-wasm-compile (or similar) as a separate npm package. The CLI lists it as an optional dependency. Users who want MariaDB run npm install @php-wasm/mariadb-wasm-compile explicitly.

  2. Download on demand — when --database=mariadb is used and the binary isn't on disk, download it from a known URL (GitHub release, npm registry, CDN). Similar to how some tools fetch platform-specific binaries at runtime. No install-time cost for users who don't need it.

  3. GitHub release artifact — attach the built mariadb.js + mariadb.wasm to a GitHub release. The CLI downloads them on first use and caches locally. Decouples the binary from npm entirely.

  4. Separate npm package + auto-download — combine 1 and 2. Publish to npm for offline/CI use, but also support downloading from npm's registry URL if the package isn't installed. The CLI checks for the local package first, falls back to downloading.

The binary doesn't change often (only when MariaDB is upgraded or WASM patches change), so any caching strategy works well here.

Test plan

  • 22 unit tests pass (npx vitest run in packages/playground/mariadb)
  • WordPress installs and serves pages on MariaDB WASM
  • No "Can't open and lock privilege tables" warnings
  • No thread stack overrun errors
  • Data files persist on host filesystem via NODEFS
  • --database=mariadb works without --mariadb-wasm-module (uses in-repo default)

adamziel added 3 commits April 7, 2026 21:46
WordPress Playground has always used SQLite as its database engine. This
commit introduces an alternative: a MariaDB server compiled to WebAssembly,
running in-process alongside PHP.

A new @wp-playground/mariadb package wraps the mariadb-wasm Emscripten
module with a clean JavaScript bridge and implements enough of the MySQL
wire protocol over TCP for PHP's mysqli extension to connect. The CLI
gains --database=mariadb and --mariadb-wasm-module flags. When selected,
the server starts before worker threads spawn, and WordPress gets real
MySQL credentials in wp-config.php instead of the SQLite integration plugin.
22 tests covering the MariaDBBridge (C API wrapping, init/destroy
lifecycle, query execution, error handling) and the MySQL protocol
server (TCP handshake, COM_QUERY for SELECT and non-SELECT, COM_PING,
COM_QUIT, error packets, server shutdown).
The embedded server needs data directories in the Emscripten virtual
filesystem and specific startup flags (--skip-grant-tables, --datadir,
--default-storage-engine=MyISAM) to work without mysql_install_db.

The init() method now creates /usr/local/mysql/data in MEMFS, builds
the argv array for mysql_server_init, and passes the flags so the server
starts cleanly. This was tested end-to-end with the compiled WASM module
from github.com/adamziel/mariadb-wasm — CREATE DATABASE, CREATE TABLE,
INSERT, and SELECT all work correctly.
@adamziel adamziel requested review from a team, Copilot and zaerl April 8, 2026 11:00
@adamziel adamziel marked this pull request as draft April 8, 2026 11:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds MariaDB 11.4 (WASM) as an alternative database backend for WordPress Playground CLI, enabling WordPress to connect via MySQL wire protocol instead of the SQLite integration plugin.

Changes:

  • Introduces new @wp-playground/mariadb package (WASM bridge + MySQL protocol TCP server) with Vitest coverage and NX/Vite build config.
  • Extends Playground CLI with --database=mariadb and --mariadb-wasm-module, starting the embedded MariaDB server before spawning PHP workers.
  • Updates WordPress boot to persist runtime DB constants into wp-config.php for pre-boot credential checks.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
tsconfig.base.json Adds TS path alias for the new @wp-playground/mariadb package.
packages/playground/wordpress/src/boot.ts Writes DB constants into wp-config.php when provided at runtime.
packages/playground/mariadb/* New MariaDB WASM bridge, MySQL protocol server, build/test config, and package metadata.
packages/playground/cli/src/run-cli.ts Adds CLI flags, starts/stops MariaDB WASM server, passes port to workers.
packages/playground/cli/src/cli-output.ts Prints selected DB engine in the CLI summary.
packages/playground/cli/src/blueprints-v1/blueprints-v1-handler.ts Injects MySQL credentials/constants for MariaDB into worker options.
packages/playground/cli/project.json Adds implicit dependency on playground-mariadb.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +104 to +112
let buf = Buffer.alloc(0);
function onData(data: Buffer) {
buf = Buffer.concat([buf, data]);
// Give it a moment to collect all packets for multi-packet responses
setTimeout(() => {
socket.removeListener('data', onData);
resolve(buf);
}, 50);
}
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fixed setTimeout(50) makes tests timing-dependent and can be flaky on slower CI runners. Prefer reading deterministically: parse packets incrementally and resolve when you’ve received the expected terminator packet (e.g., EOF/OK/Error) or a known number of packets for the command.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 50ms works because the embedded server responds synchronously (same event loop tick), but agreed it's fragile for slow CI. Will improve with deterministic packet parsing in a follow-up.

return Buffer.concat(parts);
}

describe('MySQL Protocol Server', () => {
Copy link

Copilot AI Apr 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The protocol server implements COM_INIT_DB and COM_FIELD_LIST (and DB selection during handshake), but the spec file currently doesn't assert those behaviors. Add tests that: (1) send a handshake response with CLIENT_CONNECT_WITH_DB and verify the bridge sees USE/CREATE DATABASE, (2) send COM_INIT_DB and verify it switches DB and returns OK, and (3) send COM_FIELD_LIST and verify an EOF packet response.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These paths are covered end-to-end (WordPress connects, selects databases, runs queries), but explicit unit tests for COM_INIT_DB and CLIENT_CONNECT_WITH_DB would be valuable. Will add in a follow-up.

adamziel added 9 commits April 8, 2026 14:35
MariaDB's Aria storage engine is stubbed for WASM but the query
optimizer still tries to use it for internal temp tables. Switch
the default-tmp-storage-engine to MyISAM and add loose-aria flag
to prevent Aria temp table creation attempts.
The MariaDB C API returns unsigned long long values for affected_rows,
insert_id, and field metadata (length, flags). Without WASM_BIGINT,
Emscripten truncates these to signed 32-bit, causing negative values
that crash Node.js Buffer operations.

The fix: clamp negative values to 0 for affected_rows/insert_id, and
use unsigned interpretation (>>> 0) for field metadata. The protocol
server's encodeLenEncInt also guards against negative inputs.

This was the final blocker for WordPress installation — with these
fixes, WordPress installs and runs successfully on MariaDB WASM.
MariaDB in WASM can't detect the Emscripten stack size, so
thread_stack defaults to 0 and large INSERT queries fail with
"Thread stack overrun." Setting it to 1MB fixes this.

Also suppress the "Can't open and lock privilege tables" warnings
that are expected when running with --skip-grant-tables.
The embedded server logged "Can't open and lock privilege tables"
because the mysql system database was empty — no tables existed.
This is the equivalent of never running mysql_install_db.

The init() method now creates the essential system tables
(global_priv, plugin, servers, func, proc) with a root user that
has all privileges. This eliminates the privilege table warnings.

Also set --thread-stack=1048576 because Emscripten can't detect
the WASM stack size, so MariaDB defaults thread_stack to 0 and
rejects large INSERT queries with "Thread stack overrun."
Instead of disabling 64-bit integers globally (WASM_BIGINT=0), keep
proper 64-bit support and convert BigInt to Number at the bridge
boundary. mysql_affected_rows and mysql_insert_id return unsigned
long long which becomes BigInt with WASM_BIGINT=1 — the wrapBigInt
helper converts these to plain Numbers for downstream use in Buffer
operations and the MySQL wire protocol encoder.
Instead of keeping MariaDB data in Emscripten's volatile MEMFS, mount
a real host directory at the data path. The CLI creates a mariadb-data
subdir inside the shared temp directory (alongside wordpress/, tmp/,
home/) and passes it to loadMariaDBModule. The bridge mounts it via
NODEFS before mysql_server_init so all MyISAM table files (.MYD, .MYI,
.frm) land on the real filesystem.

This reuses the same temp dir infrastructure the CLI already uses for
PHP's shared directories — no new flags needed.
Brings the mariadb-wasm build into the monorepo as a first-class
package. The build.sh script compiles MariaDB 11.4 to WebAssembly
via Emscripten with all the WASM-specific patches (Aria, timers,
curses, PCRE2 toolchain, servers_init). Pre-built dist/mariadb.js
and dist/mariadb.wasm are committed so consumers don't need the
Emscripten toolchain.

The CLI now defaults to the in-repo module when --database=mariadb
is used, so --mariadb-wasm-module is optional. Running WordPress on
MariaDB is now just:

  npx nx dev playground-cli -- server --database=mariadb
Lint fixes:
- Add eslint-disable for @nx/enforce-module-boundaries in vite.config.ts
- Remove unused variables in test mocks
- Use import type for MariaDBBridge in protocol server spec

Review feedback:
- Free WASM heap allocations for argv after mysql_server_init
- Escape backticks in database names to prevent SQL injection
- Track active sockets and destroy them on server shutdown
- Store MariaDBBridge reference and call destroy() on CLI exit

Also updates package-lock.json for the two new workspace packages.
The test-built-npm-packages CI job runs package-for-self-hosting
on all packages. Without this target, @wp-playground/mariadb wasn't
published to the local test registry, causing a 404 when the CLI
package tried to resolve it as a dependency.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants